According to a Webster's New Collegiate Dictionary, the word "learn" means "to gain knowledge or understanding of or skill in by study, instruction, or experience". A neural network's knowledge is encoded in the strength of interconnections or weights between the neurons. In a completely connected network of N neurons there are N.sup.2 interconnection weights available that can be modified by a learning rule. The "learning" process a network is said to go through, in a similar sense to Webster's definition, refers to the mechanism or rules governing the modification of the interconnection weight values. One such learning rule is called Back-Propagation as illustrated by D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel Distributed Processing Vol. 1: Foundations Cambridge, Mass.: MIT Press 1986. (This work is herein referred to as "Rumelhart 86".) The Back-Propagation learning rule will be described and followed by a discussion of the synapse processor architecture to be used for the implementation of a learning machine. A back-propagation example, using an input/output encoder neural network, will then be illustrated. As our synapse processor architecture which is the subject of this patent is applicable for modeling other neural networks and for the execution of a more general class of parallel data algorithms, we will show in a further series of examples the implementation of a Boltzmann like machine and matrix processing with our new system.
During the detailed discussion of our inventions, we will reference other work including our own unpublished works, as mentioned above. These background literature references are incorporated herein by reference.